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ABSTRACT 


Given  a new  missile  system,  a set  of  required  capabilities,  and 
a limited  set  of  testing  resources,  we  desire  to  verify  mission 
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Overview 


At  times,  the  designer  of  a test  sequence  may  be  faced  with  a dilemma. 
Constraints  on  equipment,  time,  money,  or  other  resources  may  force  an 
abbreviated  testing  program.  Does  the  test  designer  then  select  as  a 
start  a relatively  "easy"  test  with  high  probability  of  success  but  a 
low  payoff,  or  does  he  select  a "tough"  test  with  a higher  payoff  but'  a 
lower  probability  of  success?  (Success  is  defined  as  meeting  the  objectives 
of  the  test.  This  would  vary  from  item  to  item.  However,  typical 
criteria  might  include  60%  of  the  shots  but  within  a 2 inch  circle  at  a 
range  of  200  meters;  the  truck  will  climb  a 30%  grade  when  loaded  with 
10,000  pounds;  the  mobility  of  the  tractor  will  equal  that  of  the  XM-1; 
etc.  ) 

We  first  develop  a theory  to  solve  this  decision  problem.  The  theory 
rests  heavily  on  results  from  Markovian  analysis.  This  theory  is  then 
applied  to  a practical  problem,  the  testing  of  the  ZAP  missile  system. 

Purpose 

The  purpose  of  this  report  is  to  describe  an  approach  to  the  test 
designer's  decision  problem  when  the  test  environment  can  be  modelled  as 
a Markov  process  and  when  an  "expected  reward"  decision  criterion  can  be 
used.  The  thrust  of  the  report  is  tutorial. 

The  Model 

We  assume  a set  of  N possible  discrete  test  states,  1,2,3,...,  N-1,N, 
with  the  higher  number  connoting  a more  demanding  test  with  a greater 
reward  but  a lower  probability  of  completing  the  test  successfully. 

We  also  hypothesize  that  the  progression  from  test  state  to  test 
state  satisfies  the  Markovian  assumption.  We  explicitly  state  the 
assumption  by  letting 

s(n)  = j 0<n  < °°,  l<j<N 


where  s(n)  represents  the  test  state  at  the  n 
test,  and  j is  the  identifier  of  the  state. 


th 


transition  from  test  to 


The  Markovian  Assumption  requires  that  the  probability  of  transi- 
tioning from  state  to  state  be  solely  dependent  on  the  state  presently- 
occupied.  Mathematically,  we  represent  this  as 

Pr  | s(n+l)=j  | s(n)=i  | = Pr  |s(n+l)=j  | s(n)=i,s(n-l)=k,  ..., 

s(0)  = ml 

= Pij  ’ 


(1) 


The  use  of  the  transition  probabilities  is  completely  flexible  and 
enables  the  analyst  to  capture  any  test  sequencing  decision  rules.  As 
examples,  suppose  that  the  present  state  is  state  i.  "Complete"  success 
might  call  for  state  k to  be  the  next  state;  "Partial"  success  might 
call  for  state  j to  be  the  next  state.  Similiarly,  "complete"  failure 
might  call  for  a return  to  state  1 and  "partial"  failure  to  state  2. 
Implied  in  this  description  is 

l<2<i<j<k 
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These  rules  are  described  by  the  analyst  a priori  assigning  values  to 

- Complete  failure 

p^2  - Partial  failure 

p^j  - Partial  success 

p^  - Complete  success 

We  may  particularize  this  concept  by  describing  a four  state  test  program 
as  shown  in  Figure  1.  This  figure  depicts  various  probabilities  of 
moving  from  test  state  to  test  state.  In  particular  we  note  failure  of 
test  or  state  2 moves  the  test  back  to  test  1;  failure  of  3 may  lead  to 
a repeat  of  3;  and  passing  state  2 may  lead  to  either  3 or  4. 


To  Test  State  j 


We  have  previously  noted  that  higher  rewards  accrue  for  completion  of 
higher  numbered  tests.  Therefore,  we  arbitrarily  assign  the  following 
rewards 


(3) 


The  Methodology 

The  Markov  model  has  existed  since  1907.  However,  solution  diffi- 
culties have  limited  its  application,  and  the  model,  to  a certain  extent, 
has  remained  a mathematical  curio.  The  advent  of  the  computer  has  now 
made  practical  the  matrix  manipulations  necessary  for  the  solutions. 

For  our  particular  case  we  define  a variable 


vij  (n) 

as  the  number  of  times  test  state  j is  occupied  through  n transitions 
given  that  the  process  started  in  the  state  i. 

We  need  the  expected  value  v^.  (n)  and  first  require 


(n)  = Pr  |s(n)=j  | s(0)=i| 


(4) 


or  the  probability  that  after  transition  n,  the  item  will  be  in  state  j 
conditioned  on  starting  in  state  i.  By  definition,  <f>.  ,(n)=0  for  n<0, 
<J>^.(0)=0  for  i^j,  and  <f>,  ,(0)=1  for  i=j.  J 


'ij' 


[j  (n)l= 


'ij 


♦ll(n) 


Vn> 


^(n). . 


^(n). . . 


Vn) 


*lM<n) 


(5) 
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(6) 


One  of  the  classic  results  of  Markov  theory  is  that 

[$»>]  - M ■ H ••••  W 

- tp]n 

where  [p] 0 = H , the  identity  matrix. 

The  calculations  of  the  jj[]  matrix  is  only  a means  to  the  end,  the  end 
being  the  calculation  of  V,,(n),  the  expected  state  occupancies.  We 
calculate  these  by  defining'* an  indicator  random  variable  X,,(n)  such 
that  r t J 


V”1 


if  s(n)= 


otherwise 


n)=j 

rise 


conditioned  on  s(0)  = i 


Then  the  expected  value  of  X^.  is 


X^Cn)  = 1 • (Jr^n)  + 0 • £l  - 


We  also  reason  that 


v,  ,(n)  = £ 


vA1(n)  = X)  xn 


= E 

m=0  J 


Letting 


[v(n)]  = . 


Vll(n) 


Vl2(n) 


VlN(n) 


Vn) VNN(n)| 


Kj(n) 


n 

m=0 


hi(n) 


^12^ n ^ •••• 


n ^ • • • • n ) 


■E  M" 

m=0 


The  expected  state  occupancies  for  the  example  problems  are  shown  in  Table  1. 


Although  close  to  home  we  still  have  cne  or  two  more  steps  to  complete.  The 
transitions  from  test  state  to  test  state  yield  different  rewards.  Consider 
test  state  3 as  an  example.  The  03  matrix  (2)  shows  that  30$  of  the 
transitions  are  to  state  2,  10$  to  a retest  at  3,  and  60$  to  test  state  4. 
These  probabilities  and  the  associated  rewards  are  shown  in  Table  2. 


State 

State  Probability 

Reward 

Expected  Reward 

2 

0.3 

0 

0 

3 

0.1 

2 

.2 

4 

0.6 

12 

7.2 

Total  7.4 

Table  2 


Therefore,  the  expected  reward  each  time  we  transition  out  of  state  3 is 
7.4  units. 

We  may  generate  the  entire  expected  reward  matrix,  a column  matrix,  [R], 

by 

[R]  = diag  [P]  [R]T 

where  [r]  T is  the  transpose  of  the  [r]  matrix  and  diag  implies  using  only 
the  diagonal  elements  of  the  n x n matrix  one  obtains  by  multiplying 

C P J by  L r3  • 

We  must  digress  to  be  perfectly  precise  about  "counting".  Rewards  accrue 
only  on  transitions  out  of  a state.  The  expected  state  occupancies  include 
a final  occupancy  with  no  associated  exiting.  Therefore,  the  correct 
[v(n)J  is  one  less  than  the  number  of  tests  to  be  run,  i.e.,  if  the  analyst 
is  looking  for  the  optimal  starting  state  conditioned  on  12  tests  he  is 
interested  in[V(  11 

Multiplication  of  (X)  by  (y(njj  gives  the  expected  value  from  each  state 
occupied.  We  label  this  |fe(n)Jor  the  benefit  matrix, 

[B(nJ]  = [v(n  3 [I] 

We  will  illustrate  these  abstract  concepts  with  a specific  example. 
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Start  in  1 2 3 4 

State 


1 

1.0000 

0. 

0. 

0. 

2 

0. 

1.0000 

0. 

0. 

n = 0 

3 

0. 

0. 

1.0000 

0. 

4 

0. 

0. 

0. 

1.0000 

1 

1.2000 

0.8000 

0. 

0. 

2 

0.3000 

1.0000 

0.5000 

0.2000 

n = 1 

3 

0.0000 

0.3000 

1.1000 

0.6000 

4 

0.0000 

0.0000 

0.5000 

1.5000 

1 

1.4800 

0.9600 

0.4000 

0.1600 

2 

0.3600 

1.3900 

0.6500 

0.6000 

n = 2 

3 

0.0900 

0.3300 

1.5600 

1.0200 

4 

0.0000 

0.1500 

0.8000 

2.0500 

1 

1.5800 

1.3000 

0.6000 

0.5100 

2 

0.4900 

1.4800 

1.0600 

0.9700 

n = 3 

3 

0.1200 

0.5400 

1.8300 

1.5100 

4 

0. 0500 

0.2400 

1.1800 

2.5400 

1 

1.7100 

1.4500 

0.9700 

0.8800 

2 

0.5400 

1.7100 

1.3300 

1.4200 

n = 4 

3 

0.1900 

0.6400 

2.2100 

1.9600 

4 

0. 0800 

0.3900 

1.5100 

3.0200 

1 

1.7800 

1.6600 

1.2600 

1.3100 

2 

0.6200 

1.8300 

1.7000 

1.8500 

n = 5 

3 

0.2300 

0.8100 

2.5200 

2.4400 

4 

0.1300 

0.5200 

1.8600 

3.4900 

1 

1.8500 

1.8000 

1.6100 

1.7400 

2 

0.6700 

2.0100 

2.0100 

2.3100 

n = 6 

3 

0.2900 

0.9400 

2.8800 

2.8900 

4 

0. 1800 

0.6600 

2.1900 

3.9600 

I 

1 
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An  example 


A new  missile  system,  known  as  ZAP,  has  been  developed  and  is  now  * 
in  the  testing  stage.  We  are  interested  in  determining  how  the  missiles 
will  perform  and  whether  they  will  meet  established  criteria. 


A test  state  in  our  procedure  may  be  defined  as  a set  of  factors 
determining  the  environment  under  which  a missile  is  fired.  Each  factor 
has  two  levels,  an  easy  condition  and  a difficult  condition.  We  will 
first  look  at  a simple  model  where  the  outcome  of  each  experiment  is 
only  hit  or  miss.  For  the  missile  system  under  consideration  the  factors 
and  conditions  are  as  follows: 

Factor 

Factor  # 

Condition 

Target  speed 

1 

Slow,  fast 

Target  altitude 

2 

High,  low 

Maneuver 

3 

No,  yes 

IR  Countermeasures 

4 

No,  yes 

Our  4 factors  and  2 conditions  establish  2^,  or  16  test  states. 
These  are  found  in  appendix  1.  We  can  further  structure  these  states 
into  five  "macro"  levels.  Level  1 has  all  factors  in  the  easiest  con- 
dition; level  2 has  each  state  with  one  factor  at  the  difficult  condi- 
tion; level  3 has  two  difficult  factors;  level  4 has  three  difficult 
factors,  and  level  5 has  all  factors  at  maYiimmi  difficulty. 

A transition  diagram  showing  possible  paths  can  be  developed  for 
this  scenario  such  as  the  one  shown  in  Appendix  2.  Missiles  are  fired 
individually  in  accordance  with  the  following  rules:  if  a test  is 
successful,  follow  path  A.  If  unsuccessful,  follow  path  B.  Using  a 
combination  of  historical  data  on  somewhat  similar  missile  systems,  and 
the  knowledge  of  technical  experts,  probabilities  for  each  transition 
were  assessed.  The  [p]  matrix  is  shown  at  Appendix  3. 

By  again  consulting  the  technical  experts,  a relative  "benefit" 
scale  for  passing  tests  of  varying  degrees  of  difficulty  was  established 
and  is  shown  at  Appendix  4. 

Using  the  results  developed  earlier,  we  then  calculate  the  proba- 
bilities of  reaching  any  state  as  a function  of  the  number  of  missiles 
tested  and  the  initial  test  state.  For  this  example  the  expected  value 
results  yielded  a decision  rule  far  different  than  the  testing  program 
currently  in  use.  At  present,  testing  begins  at  the  lowest  levels  and 
becomes  progressively  more  difficult.  Our  model  suggests  starting  the 
testing  process  in  macro-level  4.  Within  this  level,  test  state  12 
proved  to  be  a slightly  better  choice.  This  was  true  regardless  of  the 
number  of  missiles  available  to  be  fired. 
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Thus  far,  we  have  still  only  discussed  the  benefits  of  passing 
tests  with  no  consideration  of  costs.  Fortunately,  this  can  easily  be 
incorporated  in  the  model.  Actual  costs  have  been  approximated  by  our 
technical  experts  and  the  following  information  is  provided: 

1)  a target  presentation  costs  $1000 

2)  loss  of  target  (physical  intercept  or  accident)  is  $82,000. 

3 ) a success  may  be  a technical  kill  ( target  recoverable ) or  a 
physical  intercept  (target  lost) 

4)  one  out  of  three  targets  hit  is  a physical  intercept 

5)  If  a target  is  missed,  not  fired  at,  or  a technical  kill,  there 
is  a 9555  chance  of  recovery. 

6)  refurbishing  costs  of  recovered  targets  are  $7000. 

We  can  portray  the  expected  cost  picture  as  shown  in  figure  2. 


.05 


(costs  are  in  thousands) 


FIGURE  2 

Since  p is  determined  by  test  state,  we  can  derive  the  expression  for 
expected  cost  as  a function  of  test  state  from  Figure  2 as: 

<Expected  cost | test  state>  = 35.5kp  + 11.75k  (l-p) 

= 11.75k  - 23.75kp 
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Using  a cost  oriented  decision  rule  we  again  evaluated  the  optimal 
starting  test  state.  Once  again  we  found  that  macro-level  4 is  the 
place  to  start  since  costs  are  the  cheapest.  Within  level  4,  state  15 
had  a slight  edge.  Again,  this  was  independent  of  number  of  missiles 
tested. 

To  make  a decision  based  upon  both  cost  and  benefit,  we  used  a 
benefit/cost  ratio  as  the  value  input  to  the  model  and  again  calculated 
expected  utilities.  The  results  were  the  same  macro-level  4 was  best  on 
a maximum  B/C  ratio  standard,  with  state  12  being  slightly  dominant.  The 
results  for  specified  numbers  of  missiles  are  shown  for  all  three  decision 
criteria  in  Appendix  5. 

We  next  looked  at  a slightly  different  testing  scheme.  Two  missiles 
instead  of  one  are  fired  in  each  test  environment.  The  possible  outcomes 
are  now  2 misses,  1 hit  and  1 miss,  or  2 hits.  The  results  of  each  test 
determined  what  the  next  test  state  will  be.  The  transition  diagram  is 
shown  at  Appendix  6.  If  two  missiles  hit,  follow  path  A,  if  one  hits, 
follow  path  B,  and  if  none  hit,  follow  path  C.  The[p]matrix  for  this 
diagram  is  at  Appendix  7.  The  same  benefit  scale  was  used. 


Considering  the  same  costs  as  before,  the  expected  cost  picture  is  now 
shown  in  Figure  3. 


From  this  we  can  express  expected  cost  as  a function  of  test  state  as: 

<Expected  Cost | Test  State>  = 71kp2  + 23.5k(l-p)2  + 47 . 25k( 2p )( 1-p ) 

= 47.5kp  + 23. 5k 

We  again  considered  three  decision  criteria  - expected  benefits,  expected 
costs,  and  expected  benefits/costs  ratio.  Results  are  summarized  in 
Appendix  8.  As  before,  the  higher  macro-levels  dominate.  Level  5 
(state  16)  was  best  in  several  cases,  with  level  4 a very  close  second. 

Realizing  that  the  probabilities  and  benefits  are  very  subjective, 
sensitivity  analysis  was  performed  to  see  if  small  changes  in  the  variables 
critically  changed  the  results.  First,  probabilities  were  adjusted  to 
"bias"  the  model  towards  lower  testing  levels.  Next,  benefits  were 
modified,  again  with  a bias  towards  lower  levels.  Finally,  both  were 
adjusted  simultaneously.  In  all  cases,  even  though  we  tried  to  force 
the  decision  rule  to  change,  it  was  highly  insensitive  to  these  modifications. 
Macro-level  4 was  the  testing  environment  that  consistently  proved  to  be 
the  best. 
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Summary  and  Conclusions 

We  have  developed  a model  to  assist  the  test  designer  in  his  choice  of 
initial  starting  point  in  a test  sequence.  The  analysis  is  dependent  on 
the  Markov  assumption  as  well  as  the  ability  of  the  tester  to: 

1.  Identify  discrete  test  states. 

2.  Enumerate  costs  and  benefits. 

3.  Determine  rules  and  a priori  probabilities  for  progression  from 
test  state  to  test  state. 

We  have  applied  this  model  to  a hypothetical  situation,  the  testing  of 

the  ZAP  missle.  We  have  conducted  a three  dimensional  sensitivity  in 

the  sense  that  we  have  varied  (l)  costs  and  benefits,  (2)  probabilities, 

and  (3)  decision  criteria.  In  each  case  we  found,  based  on  the  hypothesized 

decision  criteria,  that  starting  at  a "high"  test  state  was  optimal  or, 

to  put  our  findings  in  the  negative,  starting  at  test  state  1,  the 

"normal"  starting  point  was  never  optimal.  This  is  not  to  suggest  that 

other  test  patterns  would  never  logically  start  in  state  1.  However, 

for  the  ZAP  missle,  over  a wide  range  of  variables,  a linear  test  progression 

yields  expected  rewards  substantially  less  than  starting  in  a "high" 

state. 

The  model  and  the  method  of  analysis  we  have  used  is  quite  flexible.  We 
can  easily  expand  the  number  of  test  states  and  change  the  rules  for 
progression  from  state  to  state.  We  can  calculate  the  variance  of  the 
expected  reward  or  even  produce  the  entire  probability  distribution  for 
each  starting  state.  Finally,  we  can  include  the  decision  maker's  risk 
aversion  if  the  criteria  of  expected  value  is  considered  extreme. 

We  feel  the  economics  possible  through  this  altered  procedure  merit  a 
full  scale  field  test. 


Appendix  1 
Factors! 


State 

Speed 

Altitude 

Maneuver 

1RCM 

1 

slow 

low 

no 

no 

2 

slow 

low 

no 

yes 

3 

slow 

low 

yes 

no 

4 

slow 

high 

no 

no 

5 

fast 

low 

no 

no 

6 

slow 

low 

yes 

yes 

7 

slow 

high 

no 

yes 

8 

fast 

low 

no 

yes 

9 

slow 

high 

yes 

no 

10 

fast 

low 

yes 

no 

11 

fast 

high 

no 

no 

12 

slow 

high 

yes 

yes 

13 

fast 

low 

yes 

yes 

14 

fast 

high 

no 

yes 

15 

fast 

high 

yes 

no 

16 

fast 

high 

yes 

yes 
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Appendix  2 

TRANSITION  DIAGRAM  - CASE  I 


TRANSITION  MATRIX  - CASE  I 


i 


APPENDIX  5 


NUMBER  OF  MISSILES  AVAILABLE  FOR  TEST  PROGRAM 

8 12  16 


Benefit 

Cost 

B/C 

Ratio 

Benefit 

Cost 

B/C 

Ratio 

Benefit 

Cost 

B/C 

Ratio 

1 

u&gm 

.29 

.33 

146.4 

422.6 

2 

69.0 

209.7 

.33 

110.7 

312.0 

.35 

152.5 

414.2 

.37 

3 

67.1 

212.3 

.35 

108.8 

314.6 

.346 

150.6 

416.8 

.361 

4 

67.8 

216.2 

.314 

109.5 

318.4 

.344 

151.3 

420.6 

.360 

5 

64.5 

217.5 

.297 

106.1 

319.8 

.332 

147.9 

422.0 

.35 

6 

81.0 

205.8 

.394 

122.8 

308.1 

.399 

164.6 

410.3 

.401 

7 

79.6 

207.3 

.384 

121.3 

309.5 

.392 

163.1 

411.7 

.396 

8 

78.2 

209.2 

.373 

119.9 

311.4 

.385 

161.8 

413.6 

.391 

9 

77.7 

206.6 

.376 

119.5 

308.9 

.387 

161.3 

411.1 

.392 

10 

79.8 

210.4 

.379 

121.6 

312.6 

.389 

163.4 

414.8 

.394 

11 

75.8 

210.9 

.359 

117.6 

313.1 

.376 

159.4 

415.3 

.383 

12 

84.3 

204.1 

.413 

126.1 

306.4 

.412 

167.9 

408.6 

.411 

13 

84.0 

204.9 

.410 

125.8 

307.1 

.410 

167.7 

409.3 

.409 

14 

82.4 

203.8 

.404 

124.2 

306.0 

.406 

166.0 

408.2 

.407 

15 

79.7 

201.0 

.397 

121.5 

303.2 

.401 

163.3 

405.5 

.403 

16 

83.6 

204.9 

.408 

125.4 

307.1 

.408 

167.2 

409.3 

.409 

(THOUSANDS) 


Appendix  6 

TRANSITION  DIAGRAM  - CASE  II 


O'  V* 
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TRANSITION  MATRIX  - CASE  II 


2 

44.0 

221.7 

.148 

76.9 

326.4 

.236 

111.9 

426.7 

.262 

3 

44.7 

226.0 

.148 

78.4 

327.1 

.240 

113.6 

425.5 

.267 

4 

46.1 

224.6 

.205 

80.0 

323.4 

.247 

114.9 

419.8 

.274 

5 

46.7 

219.6 

.212 

80.1 

316.9 

.253 

114.2 

411.2 

.278 

6 

69.6 

210.3 

.331 

105.3 

311.3 

.338 

141.4 

410.0 

.345 

7 

70.6 

212.4 

.332 

106.7 

312.1 

.342 

142.8 

410.1 

.348 

8 

71.4 

213.7 

.334 

107.6 

312.7 

.344 

144.0 

410.6 

.351 

9 

68.2 

213.6 

.319 

104.4 

312.2 

.334 

140.6 

409.8 

.343 

10 

70.5 

213.9 

.329 

107.0 

312.1 

.343 

143.2 

409.4 

.350 

11 

69.6 

210.6 

.331 

105.3 

306.8 

.343 

140.9 

402.5 

.350 

12 

78.7 

197.7 

.398 

115.6 

296.4 

.390 

152.4 

394.7 

.386 

13 

79.0 

198.7 

.397 

115.7 

296.9 

.389 

152.3 

394.9 

.385 

14 

75.8 

197.2 

.384 

112.6 

295.3 

.381 

149.1 

393.1 

.379 

15 

72.5 

195.8 

.370 

109.1 

293.7 

.371 

145.6 

391.3 

.372 

16 

74.2 

185.2 

.400 

117.3 

284.0 

.392 

148.0 

382.2 

.387 

( THOUSANDS ) 


